A graphics processing unit (GPU) processes three-dimensional (3-D) graphics using a graphics pipeline formed of a sequence of programmable shaders and fixed-function hardware blocks. For example, a 3-D model of an object that is visible in a frame can be represented by a set of triangles, other polygons, or patches which are processed in the graphics pipeline to produce values of pixels for display to a user. The triangles, other polygons, or patches are collectively referred to as primitives. The process includes mapping textures to the primitives to incorporate visual details that have a higher resolution than the resolution of the primitives. The GPU includes a dedicated memory that is used to store texture values so that the texture values are available for mapping to primitives that are being processed in the graphics pipeline. Textures can be stored on a disk or procedurally generated as they are needed by the graphics pipeline. The texture data stored in the dedicated GPU memory is populated by loading the texture from the disk or procedurally generating the data. The dedicated GPU memory is typically a relatively small memory, which limits the amount of texture data that can be stored in the dedicated GPU memory. Furthermore, the overhead required to populate the texture data can be significant, particularly if only a small subset of the texture data is used to render the final screen image.